Algorithmic Decision-making Systems Research Articles

Identifying Reasons for Bias: An Argumentation-Based Approach

As algorithmic decision-making systems become more prevalent in society, ensuring the fairness of these systems is becoming increasingly important. Whilst there has been substantial research in building fair algorithmic decision-making systems, the majority of these methods require access to the training data, including personal characteristics, and are not transparent regarding which individuals are classified unfairly. In this paper, we propose a novel model-agnostic argumentation-based method to determine why an individual is classified differently in comparison to similar individuals. Our method uses a quantitative argumentation framework to represent attribute-value pairs of an individual and of those similar to them, and uses a well-known semantics to identify the attribute-value pairs in the individual contributing most to their different classification. We evaluate our method on two datasets commonly used in the fairness literature and illustrate its effectiveness in the identification of bias.

Trust in algorithmic decision-making systems in health: A comparison between ADA health and IBM Watson.

Algorithmic decision-making systems (ADMs) support an ever-growing number of decision-making processes. We conducted an online survey study in Flanders (n = 1,082) to understand how laypeople perceive and trust health ADMs. Inspired by the ability, benevolence, and integrity trustworthiness model (Mayer et al., 1995), this study investigated how trust is constructed in health ADMs. In addition, we investigated how trust construction differs between ADA Health (a self-diagnosis medical chatbot) and IBM Watson Oncology (a system that suggests treatments for cancer in hospitals). Our results show that accuracy and fairness are the biggest predictors of trust in both ADMs, whereas control plays a smaller yet significant role. Interestingly, control plays a bigger role in explaining trust in ADA Health than IBM Watson Oncology. Moreover, how appropriate people evaluate data-driven healthcare and how concerned they are with algorithmic systems prove to be good predictors for accuracy, fairness, and control in these specific health ADMs. The appropriateness of data-driven healthcare had a bigger effect with IBM Watson Oncology than with ADA Health. Overall, our results show the importance of considering the broader contextual, algorithmic, and case-specific characteristics when investigating trust construction in ADMs.

Objectives for algorithmic decision-making systems in childhood asthma: Perspectives of children, parents, and physicians.

To identify with children, parents and physicians the objectives to be used as parameters for algorithmic decision-making systems (ADMSs) adapting treatments in childhood asthma. We first conducted a qualitative study based on semi-structured interviews to explore the objectives that children aged 8-17 years, their parents, and their physicians seek to achieve when taking/giving/prescribing a treatment for asthma. Following the grounded theory approach, each interview was independently coded by two researchers; reconciled codes were used to assess code frequency, categories were defined, and the main objectives identified. We then conducted a quantitative study based on questionnaires using these objectives to determine how children/parents/physicians ranked these objectives and whether their responses were aligned. We interviewed 71 participants (31 children, 30 parents and 10 physicians) in the qualitative study and identified seven objectives associated with treatment uptake and five objectives associated with treatment modalities. We included 291 participants (137 children, 137 parents, and 17 physicians) in the quantitative study. We found little correlation between child, parent, and physician scores for each of the objectives. Each child's asthma history influenced the choice of scores assigned to each objective by the child, parents, and physician. The identified objectives are quantifiable and relevant to the management of asthma in the short and long term. They can therefore be incorporated as parameters for future ADMS. Shared decision-making seems essential to achieve consensus among children, parents, and physicians when choosing the weight to assign to each of these objectives.

Transparency in Algorithmic Decision-making: Interpretable Models for Ethical Accountability

Concerns regarding their opacity and potential ethical ramifications have been raised by the spread of algorithmic decisionmaking systems across a variety of fields. By promoting the use of interpretable machine learning models, this research addresses the critical requirement for openness and moral responsibility in these systems. Interpretable models provide a transparent and intelligible depiction of how decisions are made, as opposed to complicated black-box algorithms. Users and stakeholders need this openness in order to understand, verify, and hold accountable the decisions made by these algorithms. Furthermore, interpretability promotes fairness in algorithmic results by making it easier to detect and reduce biases. In this article, we give an overview of the difficulties brought on by algorithmic opacity, highlighting how crucial it is to solve these difficulties in a variety of settings, including those involving healthcare, banking, criminal justice, and more. From linear models to rule-based systems to surrogate models, we give a thorough analysis of interpretable machine learning techniques, highlighting their benefits and drawbacks. We suggest that incorporating interpretable models into the design and use of algorithms can result in a more responsible and moral application of AI in society, ultimately benefiting people and communities while lowering the risks connected to opaque decision-making processes.

When AI is Perceived to Be Fairer than a Human: Understanding Perceptions of Algorithmic Decisions in a Job Application Context

This study investigates people’s perceptions of AI decision-making as compared to human decision-making within the job application context. It takes into account both favorable and unfavorable outcomes, employing a 2 × 2 experimental design (decision-making agent: AI algorithm vs. human; outcome: favorable vs. unfavorable). Upon evaluating a job seeker’s suitability for a position, participants viewed algorithmic decisions as fairer, more competent, more trustworthy, and more useful than those made by humans. Interestingly, when a candidate was deemed unsuitable for hiring, people reacted more negatively to the verdict given by a human than to the same judgment offered by AI. Moreover, participants credited algorithmic decisions with greater sensitivity to both quantitative and qualitative qualifications, thus indicating algorithmic appreciation. Our findings shed light on the psychological basis of perceptions surrounding Algorithmic Decision-Making (ADM) and the responses to the decisions rendered by ADM systems.

Path To Gain Functional Transparency In Artificial Intelligence With Meaningful Explainability

Artificial Intelligence (AI) is rapidly integrating into various aspects of our daily lives, influencing decision-making processes in areas such as targeted advertising and matchmaking algorithms. As AI systems become increasingly sophisticated, ensuring their transparency and explainability becomes crucial. Functional transparency is a fundamental aspect of algorithmic decision-making systems, allowing stakeholders to comprehend the inner workings of these systems and enabling them to evaluate their fairness and accuracy. However, achieving functional transparency poses significant challenges that need to be addressed. In this paper, we propose a design for user-centered compliant-by-design transparency in transparent systems. We emphasize that the development of transparent and explainable AI systems is a complex and multidisciplinary endeavor, necessitating collaboration among researchers from diverse fields such as computer science, artificial intelligence, ethics, law, and social science. By providing a comprehensive understanding of the challenges associated with transparency in AI systems and proposing a user-centered design framework, we aim to facilitate the development of AI systems that are accountable, trustworthy, and aligned with societal values.

Social context of the issue of discriminatory algorithmic decision-making systems

Social context of the issue of discriminatory algorithmic decision-making systems

Labour market digitalization and social class: evidence of mobility and reproduction from a European survey of online platform workers

Abstract The type of work we do as adults is significantly influenced by our parents’ social class. However, digital technologies are transforming the way labour markets work. Candidates are screened using algorithmic decision-making systems. Skills are validated with online tests and feedback ratings. Communications take place online. Could these transformations undermine the advantages that have accrued to workers with privileged backgrounds or reproduce this privilege through digital divides? We examine this question with survey evidence from the online (remote) platform economy, a labour market segment where these digital transformations have progressed furthest (N = 1,001). The results reveal that online platform workers come predominantly from privileged class backgrounds, but we find less evidence of parental class shaping what types of online work they do. We conclude that digital transformations of labour markets may reproduce disparities in access to work but attenuate some class-based differences in the selection of workers by employers.

The public perceptions of algorithmic decision-making systems: Results from a large-scale survey

The public perceptions of algorithmic decision-making systems: Results from a large-scale survey

Managing unintended consequences of algorithmic decision-making: The case of Robodebt

In 2016, the Australian welfare agency Centrelink implemented an information system to automate the identification and recollection of welfare overpayments. Such algorithmic decision-making systems are increasingly leveraged to improve the efficiency of public administration. However, Centrelink’s scheme went horribly wrong: the system, branded as “Robodebt” by the popular media, generated debt notices that were inaccurate and based on insufficient evidence. Numerous vulnerable citizens who received the debt notices suffered a great deal of distress. While public controversy ensued, the ruling government continued to defend the flawed system until a court decision ruled it unlawful in 2019. This teaching case challenges one to analyze what went wrong in the implementation and management of the Robodebt system through the lens of sociotechnical systems.

When Algorithms Err : Differential Impact of Early vs. Late Errors on Users’ Reliance on Algorithms

Errors are a natural part of predictive algorithms, but may discourage users from relying on algorithms. We conduct two experiments to demonstrate that reliance on a predictive algorithm following a substantial error is affected by (i) when the error occurs and (ii) how the algorithm is used in the decision-making process. We find that the impact of an error on reliance depends on whether the error occurs early (i.e., when users first start using the algorithm) or late (i.e., after users have used the algorithm for an extended period). While an early error results in substantial and persistent reliance reduction, a late error affects reliance only temporarily and to a lesser extent. However, when users have more control over how to use the algorithm’s predictions, error timing ceases to have a significant impact. Our work advances the understanding of algorithm aversion and informs the practical design of algorithmic decision-making systems.

Outcome-Explorer: A Causality Guided Interactive Visual Interface for Interpretable Algorithmic Decision Making.

The widespread adoption of algorithmic decision-making systems has brought about the necessity to interpret the reasoning behind these decisions. The majority of these systems are complex black box models, and auxiliary models are often used to approximate and then explain their behavior. However, recent research suggests that such explanations are not overly accessible to lay users with no specific expertise in machine learning and this can lead to an incorrect interpretation of the underlying model. In this article, we show that a predictive and interactive model based on causality is inherently interpretable, does not require any auxiliary model, and allows both expert and non-expert users to understand the model comprehensively. To demonstrate our method we developed Outcome Explorer, a causality guided interactive interface, and evaluated it by conducting think-aloud sessions with three expert users and a user study with 18 non-expert users. All three expert users found our tool to be comprehensive in supporting their explanation needs while the non-expert users were able to understand the inner workings of a model easily.

Accountable Algorithms: Adopting the Public Law Toolbox Outside the Realm of Public Law

Abstract It is well known that artificial intelligence, especially machine learning, has the potential for hugely beneficial impacts on all areas of life, but also carries with it dangers such as lack of transparency, over-rigidity of decision-making, negative feedback loops and unreasonable inferences. Such dangers also have the potential to be scaled across a whole area of decisions. There is, therefore, an increasing sense that we need greater accountability of algorithmic decision-making systems and it is argued here that public law and the grounds of judicial review are a ready-made toolkit specifically designed to render decision-makers accountable and to make precisely the kinds of trade-offs we will need to make between effectiveness and efficiency on the one hand and fairness on the other. These tools can therefore provide a blueprint as we work out how to govern ADM and render it accountable, even in a private context. This can take place by using the public law toolkit in interpreting existing legislation, informing future regulation and even through the exercise of a common law supervisory jurisdiction.

Public procurement of artificial intelligence systems: new risks and future proofing.

Public entities around the world are increasingly deploying artificial intelligence (AI) and algorithmic decision-making systems to provide public services or to use their enforcement powers. The rationale for the public sector to use these systems is similar to private sector: increase efficiency and speed of transactions and lower the costs. However, public entities are first and foremost established to meet the needs of the members of society and protect the safety, fundamental rights, and wellbeing of those they serve. Currently AI systems are deployed by the public sector at various administrative levels without robust due diligence, monitoring, or transparency. This paper critically maps out the challenges in procurement of AI systems by public entities and the long-term implications necessitating AI-specific procurement guidelines and processes. This dual-prong exploration includes the new complexities and risks introduced by AI systems, and the institutional capabilities impacting the decision-making process. AI-specific public procurement guidelines are urgently needed to protect fundamental rights and due process.

Algorithmic harms and digital ageism in the use of surveillance technologies in nursing homes.

Ageism has not been centered in scholarship on AI or algorithmic harms despite the ways in which older adults are both digitally marginalized and positioned as targets for surveillance technology and risk mitigation. In this translation paper, we put gerontology into conversation with scholarship on information and data technologies within critical disability, race, and feminist studies and explore algorithmic harms of surveillance technologies on older adults and care workers within nursing homes in the United States and Canada. We start by identifying the limitations of emerging scholarship and public discourse on “digital ageism” that is occupied with the inclusion and representation of older adults in AI or machine learning at the expense of more pressing questions. Focusing on the investment in these technologies in the context of COVID-19 in nursing homes, we draw from critical scholarship on information and data technologies to deeply understand how ageism is implicated in the systemic harms experienced by residents and workers when surveillance technologies are positioned as solutions. We then suggest generative pathways and point to various possible research agendas that could illuminate emergent algorithmic harms and their animating force within nursing homes. In the tradition of critical gerontology, ours is a project of bringing insights from gerontology and age studies to bear on broader work on automation and algorithmic decision-making systems for marginalized groups, and to bring that work to bear on gerontology. This paper illustrates specific ways in which important insights from critical race, disability and feminist studies helps us draw out the power of ageism as a rhetorical and analytical tool. We demonstrate why such engagement is necessary to realize gerontology's capacity to contribute to timely discourse on algorithmic harms and to elevate the issue of ageism for serious engagement across fields concerned with social and economic justice. We begin with nursing homes because they are an understudied, yet socially significant and timely setting in which to understand algorithmic harms. We hope this will contribute to broader efforts to understand and redress harms across sectors and marginalized collectives.

Enhancing Fairness Perception – Towards Human-Centred AI and Personalized Explanations Understanding the Factors Influencing Laypeople’s Fairness Perceptions of Algorithmic Decisions

Whether we like it or not, algorithmic decision-making systems (ADMSs) are all around us. These systems assist both public institutions and private organizations in making decisions that exert a significant impact on our lives. The widespread use of artificial intelligence (AI) and machine learning (ML) systems and the potential risks of using them are the subjects of intensive, ongoing research. It is imperative to ensure their fairness and transparency. The understanding that ADMSs should be subject to human supervision and examined for laypeople’s perceived fairness is clear. Laypeople’s perceptions regarding ADMSs’ fairness, their understanding of the reasons underlying the systems’ outcome (decision), and their comprehension of the linkage between the explanations and the results, influence their willingness to trust the systems, use them and accept their decisions. To determine and better understand which factors affect laypeople’s perceptions of the fairness of algorithmic decisions, we conducted an online between-subject experiment, employing a case study of a simulated AI-based recruitment decision-support system. We focused on three aspects: system characteristics (SC), personality characteristics (PC), and demographic characteristics (DC). We conducted an in-depth analysis to determine which explanation increases the perceived fairness the most. Based on the results, we suggest a framework for predicting a layperson’s perception of the fairness of the explanations. Our findings may help in understanding how to involve humans in the development and evaluation process of ADMSs, how to create personalized explanations based on the SC as well as on users’ PC and DC, and, consequently, how to enhance laypeople’s fairness perceptions regarding ADMSs.

Algorithms, leadership, and morality: why a mere human effect drives the preference for human over algorithmic leadership

Algorithms are increasingly making decisions in organizations that carry moral consequences and such decisions are considered to be ordinarily made by leaders. An important consideration to be made by organizations is therefore whether adopting algorithms in this domain will be accepted by employees and whether this practice will harm their reputation. Considering this emergent phenomenon, we set out to examine employees’ perceptions about (a) algorithmic decision-making systems employed to occupy leadership roles and make moral decisions in organizations, and (b) the reputation of organizations that employ such systems. Furthermore, we examine the extent to which the decision agent needs to be recognized as “merely” a human, or whether more information is needed about the decision agent’s moral values (in this case, whether it is known that the human leader is humble or not) to be preferred over an algorithm. Our results reveal that participants in the algorithmic leader condition—relative to those in the human leader and humble human leader conditions—perceive the decision made to be less fair, trustworthy, and legitimate, and this in turn produces lower acceptance rates of the decision and more negative perceptions of the organization’s reputation. The human leader and humble human leader conditions do not significantly differ across all main and indirect effects. This latter effect strongly suggests that people prefer human (vs. algorithmic) leadership primarily because they are human and not necessarily because they possess certain moral values. Implications for theory, practice, and directions for future research are discussed.

On mapping values in AI Governance

We propose here a conceptual framework by which to analyze legal-regulatory problematics of algorithmic decision-making systems, focusing on mechanisms of value production in their design and deployment. An aim of our intervention is to develop an investigative model for application to algorithmic decision systems with regulatory effects, including predictive artificial intelligence applications and recommender systems that filter data and suggest courses of action. Technical systems that integrate complex algorithmic techniques perform critical and sensitive functions that are both object and instrument of regulatory governance, functions such as predicting behavior, steering information flows, assessing risk, etc. These functions, however, are not simple or static phenomena, but rather contextual, partial performances of complex socio-technical dynamics. One of our interests is to discern what is valorized in this new regulatory ecology. Accordingly, we are sketching a framework to target terms and tokens of value as they are produced, reproduced, incorporated, and translated among design processes, legal practices and background conditions structuring their use. Rather than asking which values AI should satisfy in contested governance contexts, we address conceptually prior questions concerning how values manifest and ‘map’ among context-sensitive computational and social processes in the first place. Furthermore, current research often takes for granted that an AI application is produced against the backdrop of a stable and pre-defined set of values and legal practices. Existing research does not yet adequately account for the ways in which laws and values as produced in and through the ecology of the AI application differ from idealized presuppositions assumed to preexist development of the latter. For the purpose, our contribution engages three broad lines of inquiry: one, we take forward calls for a materialized study of law, such as put forward broadly by Alain Pottage, and as put forward more recently and specifically with respect to computational technologies by Mireille Hildebrandt, among others; two, we contribute to the elaboration of a critical practice for AI, in the tradition of Philip Agre; and three, our attention to assemblages potentially contributes to debates over techno-regulation or regulation by design.

Discrimination, Bias, Fairness, and Trustworthy AI

In this study, we analyze “Discrimination”, ”Bias”, “Fairness”, and “Trustworthiness” as working variables in the context of the social impact of AI. It has been identified that there exists a set of specialized variables, such as security, privacy, responsibility, etc., that are used to operationalize the principles in the Principled AI International Framework. These variables are defined in such a way that they contribute to others of more general scope, for example, the ones studied in this study, in what appears to be a generalization–specialization relationship. Our aim in this study is to comprehend how we can use available notions of bias, discrimination, fairness, and other related variables that will be assured during the software project’s lifecycle (security, privacy, responsibility, etc.) when developing trustworthy algorithmic decision-making systems (ADMS). Bias, discrimination, and fairness are mainly approached with an operational interest by the Principled AI International Framework, so we included sources from outside the framework to complement (from a conceptual standpoint) their study and their relationship with each other.

Facets of algorithmic literacy: Information, experience, and individual factors predict attitudes toward algorithmic systems

Algorithmic decision-making systems are ubiquitous in digital media, but the public has been largely unable to negotiate the role of algorithms in society. Building from the concept of attitude-behavior consistency for political behavior, we develop a framework for fostering algorithmic literacy to develop well-informed attitudes toward algorithms. As algorithms are increasingly relevant to broad societal effects, an integrative approach is needed for a full account of how the public makes sense of algorithms and their role in society. We designed and tested a novel intervention that combines algorithmic literacy with personalized user experiences to see how each component influenced attitudes toward algorithms. We found these methods jointly informed attitudes, but the intervention’s efficacy was dependent on participants’ individual differences in technology use.